ABSTRACT
Objective Taking three-dimensional (3D) motion capture system (MoCap) as the gold standard, a deep learning fusion model based on bi-lateral long short-term memory (BiLSTM) recurrent neural network and linear regression algorithm was developed to reduce system error of the Kinect sensor in lower limb kinematics measurement. Methods Ten healthy male college students were recruited for gait analysis. The 3D coordinates of the reflective markers and the lower limb joint centers were simultaneously collected using the MoCap system and the Kinect V2 sensor, respectively. The joint angles of lower limbs were calculated using the Cleveland clinic kinematic model and the Kinect kinematic model, respectively. The dataset was constructed using the MoCap system as the target and the angles via the Kinect system as the input. A BiLSTM network and a linear regression model for all lower limb angles were developed to obtain the refined angles. A leave-one subject-out cross-validation method was employed to study the performance of the models. The coefficient of multiple correlations (CMC) and root mean square error (RMSE) were used to investigate the similarity and the mean deviation between the joint angle waveforms via the MoCap and the Kinect system. ResultsIn comparison with the linear regression algorithm, the BiLSTM had better performance in the aspect of dealing highly nonlinear regression problems, especially for hip flexion/extension, hip adduction/abduction, and ankle dorsi/plantar flexion angles. The deep learning refined model significantly reduced the system error of Kinect. The mean RMSEs for all joint angles were mainly smaller than 10°, and the RMSEs of the hip joint were smaller than 5°. The joint angle waveforms presented very good similarity with the golden standard. The CMCs of joint angles were greater than 0.7 except for hip rotation angle. Conclusions The markerless gait analysis system based on deep learning fusion model developed in this study can accurately assess lower limb kinematics, joint mobility, walking functions, and has good prospect to be applied in clinical and home rehabilitation.